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Abstract,  The  DOD  routinely  demonstrates  its  eapability  to  develop  phenomenal  systems; 
however,  these  aeeomplishments  are  often  tarnished  by  substantial  cost  and  schedule  over¬ 
runs.  While  defense  policies  are  continually  being  revised  to  address  these  problems,  many 
believe  that  a  more  fundamental  souree  of  these  overruns  is  the  lack  of  flexibility  in  the  sys¬ 
tems  being  developed.  But  providing  justification  to  invest  in  flexibility  is  a  tough  sell,  as 
stakeholders  struggle  to  quantitatively  demonstrate  the  potential  return  on  investment.  There¬ 
fore,  this  paper  introduees  an  alternative  methodology  for  valuing  the  inherent  ability  of  a 
system  or  design  to  accommodate  ehange.  The  proposed  methodology  is  essentially  a  modifi- 
eation  of  the  current  life  eyele  model  and  is  premised  on  the  notion  that  the  need  for  eapabili- 
ty  ehanges  in  a  program  arise  in  a  stoehastic  manner  that  can  be  ineorporated  into  a 
continually  updated,  expeeted  value  model  presented  in  terms  of  total  program  cost. 

Introduction  and  Motivation 

While  the  U.S.  Department  of  Defense  (DOD)  routinely  fields  world-elass  weapons  systems, 
there  is  tremendous  opportunity  for  improving  the  aequisition  of  these  systems,  at  least  with 
respect  to  cost  and  schedule  performanee.  In  2009,  the  Government  Aeeountability  Office 
found  that  of  the  DOD’s  major  ongoing  aequisition  programs  that  provided  relevant  cost  data, 
69  percent  reported  an  inerease  in  total  acquisition  costs,  with  over  40  pereent  of  those  pro¬ 
grams  reporting  an  increase  in  aequisition  unit  costs  of  at  least  25  percent.  Moreover,  on  av¬ 
erage,  total  researeh  and  development  costs  were  42  percent  higher  than  originally  estimated 
and  systems  were  22  months  behind  sehedule.  Moreover,  the  older  the  program,  the  more 
pronouneed  the  cost  overruns  and  sehedule  delays.  Major  defense  programs  that  have  been  in 
development  more  than  15  years  have  seen  an  average  138  pereent  inerease  in  aequisition 
costs,  and  over  three  years  of  sehedule  delays  [1]. 

These  systemic  failings  are  widely  known  to  those  familiar  with  defense  aequisitions,  and 
there  is  nothing  partieularly  surprising  in  the  latest  numbers.  Nor  is  there  anything  surprising 
in  how  the  DOD  is  likely  to  respond  to  the  problem.  If  the  past  is  any  indication  of  the  future, 
then  we  will  soon  see  another  aequisition  reform  effort  spawned  and  promulgated  with  the 
expressed  intent  of  redueing  monetary  waste  and/or  improving  overall  mission  responsive¬ 
ness.  This  observation  is  not  meant  to  disparage  the  various  well-intentioned  reform  efforts 
and  the  dedicated  professionals  that  ereate  and  implement  them;  the  point  is,  rather,  that  the 
desired  improvements  are  seldom,  if  ever,  realized  [2,3,4]. 

One  possible  explanation  for  the  lack  of  effectiveness  of  these  aequisition  policies  is  that  they 
are  aimed  at  the  cause  rather  than  the  symptoms.  Eor  most  engineering  problems,  this  would 
be  exactly  the  right  approaeh.  One  time  it  is  not  is  when  the  root  eause  is  ineluetable.  When 
this  is  the  ease,  resourees  may  aetually  be  squandered  by  foeusing  on  the  eause,  and  instead 
should  be  aimed  at  how  best  to  mitigate  the  effects.  As  an  analogy,  it  is  more  sensible  to  con¬ 
struct  a  waterproof  shelter  than  to  try  to  develop  a  teehnique  for  stopping  rainfall  entirely. 
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With  respect  to  acquisition  programs,  the  metaphorical  role  of  the  inevitable  raincloud  is 
filled  by  uncertainty.  Every  major  program  must  contend  with  myriad  sources  of  uncertainty, 
to  include  the  emergence  of  new  threats,  technological  setbacks/breakthroughs,  requirement 
creep,  test  failures,  budget  fluctuations,  market  volatility,  workforce  turnover,  and,  of  course, 
new  acquisition  policies.  Regarding  the  last  item,  the  steady  barrage  of  acquisition  reform 
efforts  that  attempt  to  overcome  uncertainty  are  arguably  futile  since  uncertainty  cannot  be 
overcome.  Worse,  it  may  be  that  some  of  these  strategies  (e.g.,  requirement-driven  acquisi¬ 
tion)  contribute  to  the  development  of  point-solution  designs  that  are  ironically  less  capable 
of  responding  to  these  various  sources  of  uncertainty  when  they  do  arise,  thereby  inevitably 
wreaking  havoc  with  program  budgets  and  schedules.  So  instead  of  tilting  (or,  at  least,  in¬ 
stead  of  only  tilting)  at  the  windmill  of  uncertainty,  a  better  approach  may  be  to  accept  uncer¬ 
tainty  as  a  fact  of  life,  and  explore  how  we  can  design  systems  to  better  respond  to  it. 

While  the  definitional  landscape  related  to  a  system’s  ability  to  respond  to  uncertainty  is  large 
and  ambiguous,  the  term  most  often  associated  with  this  concept  is  flexibility  [5].  If  systems 
can  be  designed  in  such  a  way  that  they  are  able  to  more  readily  respond  to  various  sources  of 
change,  then  it  stands  to  reason  that  when  uncertainties  become  realities,  the  impact  to  the 
program  will  be  lessened.  Designing  flexibility  into  a  system,  which  paradoxically  focuses  on 
the  predictable  effect,  rather  than  the  unavoidable  cause,  may  be  vital  to  achieving  the  persis¬ 
tently  elusive  goal  of  improved  cost  and  schedule  performance. 

Background 

The  belief  that  a  system  can  be  designed  such  that  it  is  inherently  capable  of  being  modified 
more  quickly  or  more  inexpensively  when  confronted  with  change  is  of  growing  interest  to 
the  DOD.  The  Office  of  the  Secretary  of  Defense  is  currently  sponsoring  a  number  of  re¬ 
search  topics  related  to  this  notion  of  flexibility,  aimed  at  answering  questions  such  as — 

•  What  is  the  relationship  between  flexibility  and  cost,  schedule,  and  performance? 

•  How  much  does  flexibility  cost,  both  in  terms  of  money  and  other  system  tradeoffs? 

•  How  do  we  measure  the  value  of  flexibility? 

•  When,  and  to  what  extent,  should  we  invest  in  flexibility? 

Defining  Flexibility,  Even  a  cursory  examination  of  the  literary  landscape  makes  it  clear  that 
the  terminology  related  to  flexibility  is  largely  a  quagmire.  Despite  its  wide  usage  and  high 
regard,  flexibility  remains  a  distinctly  ambiguous  concept.  In  many  cases,  the  problem  ex¬ 
tends  to — and  is  exacerbated  by — the  casual  usage  of  many  of  the  other  non-traditional  sys¬ 
tem  design  parameters  (so-called  “-ilities”).  Eor  instance,  the  terms  “flexibility”  and 
“adaptability”  are  often  used  interchangeably,  or  conflated  with  descriptors  like  agility  and 
versatility. 

This  ontological  problem  is  tackled  by  the  authors  in  a  precursor  paper  [5],  where  an  exten¬ 
sive  literature  review  is  used  to  construct  a  novel  analytical  framework  for  clarifying  salient 
aspects  of  flexibility-related  terminology.  While  it  was  not  possible  to  descry  consensus  defi¬ 
nitions  from  the  literature,  we  did  identify  certain  dominant  characteristics  that  enabled  us  to 
formulate  a  clearer  and  more  consistent  view  of  certain  flexibility-related  terms.  A  summary 
of  these  definitions  is  provided  below  to  orient  the  reader  to  the  terminology  used  in  this  pa¬ 
per  (the  detailed  rationale  behind  these  definitions  may  be  found  in  the  referenced  paper). 
The  reader  is  cautioned  not  to  place  too  much  stock  in  these  definitions,  or  become  too  agi¬ 
tated  should  they  be  out  of  accord  with  the  reader’s  own  views.  Not  only  are  these  definitions 
not  intended  to  be  authoritative,  but  we  believe  the  methodology  proposed  in  this  paper  for 
valuing  flexibility  serves  to  effectively  obviate  much  of  the  distinctions  in  terminology. 
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•  Versatility:  Denotes  a  broad  range  of  capabilities',  a  versatile  system  has  many  capa¬ 
bilities,  but  may  or  may  not  be  flexible.  In  other  words,  versatility  ensures  the  system 
possesses  many  capabilities,  but  does  not  guarantee  that  existing  capabilities  may  be 
easily  enhanced  or  new  capabilities  may  be  easily  added.  Logically,  a  versatile  system 
is  likely  to  be  better  poised  to  accommodate  unforeseeable  sources  of  change. 

•  Overcapacity:  Refers  to  designing  more  capability  into  a  system  than  what  is  re¬ 
quired.  Sometimes  pejoratively  referred  to  as  “gold-plating,”  overcapacitization  in¬ 
cludes  adding  entirely  new  non-validated  capabilities  (e.g.,  ability  to  release 
munitions  from  a  reconnaissance  vehicle)  or  exceeding  required  performance 
attributes  (e.g.,  designing  for  Mach  2.5  when  the  requirement  is  Mach  2.0).  It  only 
pertains  to  foreseeable  sources  of  change. 

•  Process  Flexibility:  This  is  the  collection  of  structured  management  techniques  for 
ensuring  a  program  is  best  poised  to  respond  to  change.  It  includes  parallel  develop¬ 
ment,  management  reserve,  incremental  deliveries,  and  delayed  differentiation.  Fun¬ 
damentally,  process  flexibility  is  about  employing  sound  systems  engineering 
principles  that  allow  management  to  delay  decisions  and  keep  options  open  in  order  to 
remain  more  responsive  to  the  various  uncertainties  that  arise  in  a  program. 

•  Design  Flexibility:  Defined  as  the  degree  to  which  a  system  can  be  modified  to  in¬ 
crease  its  capability  in  response  to  foreseeable  external  change.  Fundamentally,  de¬ 
sign  flexibility  is  about  designing  a  system  in  such  a  way  that  its  capabilities  can  be 
more  easily  modified,  essentially  creating  capability  options.  Design  flexibility  can  be 
achieved  through  pre-provisioning  (i.e.,  scarring)  the  system  with  nascent  capabilities 
that  can  be  matured  to  full  implementation  at  a  fraction  of  the  cost  than  they  otherwise 
would  have  required.  Design  flexibility  can  also  be  achieved  by  broadening  the  de¬ 
sign-space  to  allow  for  competing  design  implementations  both  now  and  in  the  future. 
Note  that  unlike  overcapacity,  design  flexibility  does  not  commit  one  to  a  capability; 
instead,  it  poises  the  system  to  implement  the  capability  more  easily  at  a  later  time. 

While  this  paper  seeks  to  develop  a  methodology  capable  of  incorporating  all  these  elements, 
the  proximate  discussion  is  largely  focused  on  design  flexibility.  This  is  both  for  simplicity  of 
argument,  and  because  it  best  captures  the  most  commonly  perceived  notion  of  flexibility. 

Measuring  Flexibility.  With  respect  to  the  effort  to  quantify  the  value  of  flexibility,  there  is 
extensive  confusion  in  the  literature  regarding  the  difference  between  measuring  a  parameter 
and  measuring  the  value  of  a  parameter  [6].  In  fact,  there  have  been  a  surprising  number  of 
attempts  to  quantify  flexibility,  per  se,  but  very  few  attempts  to  quantify  the  value  of  flexibili¬ 
ty.  Unfortunately,  even  if  we  were  to  assume  that  measuring  flexibility  is  a  necessary  step¬ 
ping  stone  to  measuring  the  value  of  flexibility,  the  extant  methodologies  appear  to  be  of 
questionable  theoretical  validity,  and/or  are  too  restricted  in  their  applicability  [6,7].  For  in¬ 
stance,  some  approaches  to  measuring  flexibility  only  provide  an  arbitrary  flexibility  “score” 
[8,9,10],  which  is  generally  not  possible  to  translate  into  a  value  framework.  Similarly,  there 
appears  not  to  exist  a  methodology  for  measuring  process  flexibility,  and  those  methods  that 
might  otherwise  be  useful  for  measuring  design  flexibility  are  simply  not  applicable  to  the 
defense  sector.  Military  acquisition  is  fundamentally  different  from  private  sector  acquisition 
in  that  system  value  is  not  ascertained  based  on  profit  forecasts;  rather,  value  is  determined 
by  a  system’s  capabilities,  and  how  well  they  meet  warfighter  needs.  Further,  value  is  also 
driven  by  the  urgency  of  the  need,  which  is  not  always  predictable  a  priori.  These  types  of 
factors  are  not  considered  in  any  of  the  extant  proposed  methods  to  measure  flexibility. 

Value  Driven  Design,  If  we  are  to  assign  value  to  flexibility,  we  must  be  able  to  assign  value 
to  different  system  design  options.  Therefore,  of  key  interest  to  us  is  the  principle  of  value- 
driven  design  (VDD),  which  attempts  to  incorporate  value  metrics  into  engineering  designs. 
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VDD  is  a  movement  to  refocus  systems  engineering  processes  on  the  optimization  of  the 
overall  system  design,  vice  the  optimization  of  specific  system  performance  parameters  [11]. 

To  implement  VDD,  we  must  first  develop  a  value  model,  which  is  the  objective  function  for 
comparing  the  worth  of  one  design  to  another  [12],  Value  models  are  suitable  for  capturing 
the  upside  of  uncertainty,  departing  from  the  standard  reckoning  of  only  the  downside  uncer¬ 
tainty  via  traditional  risk  management  techniques.  Thus,  a  VDD  model  could  feasibly  reflect 
the  value  added  by  investing  in  a  more  flexible  design  because  of  the  potential  payoff  later. 
Furthermore,  if  the  value  of  flexibility  can  be  quantified  in  units  that  are  commensurable  with 
cost,  then  meaningful  cost-value  tradeoffs  can  be  made.  In  other  words,  VDD  may  help  us 
with  the  critical  task  of  directly  comparing  costs  and  benefits  by  assigning  values  to  each 
parameter  that  have  the  same  units  of  measurement  (presumably  dollars).  Then  the  best  (i.e., 
most  cost-effective)  design  is  simply  the  one  with  the  highest  expected  utility  [13]. 

Value-driven  design  is  not  yet  used  routinely  in  the  defense  industry,  but  the  underlying  prin¬ 
ciple  represents  a  potential  sea-state  change  in  how  the  DOD  would  manage  its  programs.  To 
date,  the  DOD  has  been  unable  to  accurately  and  reliably  predict  the  costs  of  weapon  systems 
that  meet  certain  requirements  over  a  given  period  of  time.  By  valuing  non-traditional  system 
characteristics  like  flexibility,  VDD  essentially  provides  a  more  efficient  and  strategic  ap¬ 
proach  to  systems  acquisition,  potentially  providing  a  more  accurate  and  integrated  assess¬ 
ment  of  true  system  cost  than  other  extant  putative  measures  of  life-cycle  cost.  Brown 
outlines  a  method  to  leverage  VDD  in  this  manner,  which  he  refers  to  as  “lifecycle  cost  under 
uncertainty”  [14].  The  notion  of  a  life  cycle  cost  metric  that  attempts  to  account  for  non- 
deterministic  events  in  a  program’s  life  cycle  is  at  the  heart  of  the  “more  flexible  approach  to 
valuing  flexibility,”  which  will  be  outlined  in  the  final  section  of  this  paper. 

Decision  Making  Under  Uncertainty,  Based  on  the  literature,  we  know  that  the  value  of 
flexibility  is  positively  correlated  to  uncertainty,  such  that  the  greater  the  uncertainty  in  the 
system,  the  greater  the  value  a  flexible  design  option  is  likely  to  have  [7,15,16,17].  But  if  we 
are  to  make  any  headway  on  quantifying  the  value  of  flexibility,  we  need  the  ability  to  make 
the  best  decision  under  conditions  of  uncertainty.  This  type  of  problem  has  been  studied  ex¬ 
tensively  in  economics.  The  economic  approach  to  this  problem  is  net  present  value  (NPV) 
analysis.  NPV  is  a  standard  method  for  determining  the  time  value  of  money.  It  takes  into 
account  the  net  cash  flow  at  a  particular  time  t,  as  well  as  the  required  rate  of  return  (i.e., 
discount  rate).  Thus,  the  expected  cash  flows  are  discounted  at  an  interest  rate  that  accounts 
for  the  time  value  of  money  as  well  as  the  project  risk.  Several  studies  use  NPV  as  part  of 
their  effort  to  quantify  flexibility,  including  [13,16,18,19,20]. 

Another  approach  is  real  options  analysis,  which  exists  at  the  intersection  of  value  and  uncer¬ 
tainty.  Economic  theory  defines  a  real  option  as  the  “right,  but  not  the  obligation  to  take  an 
action  at  a  predetermined  cost  and  at  a  predetermined  time”  [17].  Copeland  [21]  claims  that 
only  real  options  can  “provide  a  theoretically  sound  tool  for  valuing”  decision  flexibility.  In  a 
manufacturing  application,  Ajah  [22]  touts  real  options,  stating  “that  the  adoption  of  the  real 
options  approach  early  in  the  conceptual  design  process  can  offer  to  the  designer,  extra  de¬ 
grees  of  freedom  of  systematically  considering  and  designing  system  elements.” 

Research  Gaps 

NPV.  In  general,  researchers  tend  not  to  be  in  favor  of  using  NPV  for  decisions  involving 
flexibility  [6].  While  NPV  is  sufficient  in  cases  of  “low  uncertainty,  or  [when]  you  have  no 
scope  to  change  course”  [21,23],  it  is  not  appropriate  for  situations  involving  great  uncertain¬ 
ty,  as  it  assumes  a  predetermined  path  through  an  established  set  of  alternatives.  This  is  anti¬ 
thetical  to  the  core  aim  of  flexible  processes  and  designs,  so  a  different  method  is  needed — 
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one  that  can  take  more  (and  fewer  predetermined)  decision  options  into  account  [12,24,25]. 

Real  Options.  While  real  options  analysis  is  widely  seen  as  preferable  to  NPV,  it  comes  with 
its  own  set  of  criticisms.  For  example,  the  underlying  financial  model  (known  as  Black- 
Scholes)  is  generally  only  valid  under  certain  constraining  assumptions,  many  of  which  are 
not  likely  to  be  applicable  to  defense  acquisition.  One  of  the  model  assumptions  is  that  the 
valued  asset  must  be  traded  on  an  “efficient”  market,  where  there  is  no  possibility  of  arbi¬ 
trage.  While  arguably  true  in  the  broader  capital  market,  this  assumption  is  not  warranted 
within  the  DOD,  where  markets  are  often  artificial,  and  far  from  efficient.  Another  stipulation 
of  the  Black-Scholes  model  is  that  the  asset  must  have  a  price  that  follows  geometric  Brow¬ 
nian  motion,  thus  creating  a  return  on  the  asset  that  is  consistent  with  a  random  lognormal 
distribution.  However,  random  fluctuation  of  price  is  a  debatable  premise  in  a  standard  open 
market,  let  alone  in  the  cloistered  defense  industry.  Finally,  real  options  in  military  acquisi¬ 
tion  programs  are  likely  to  be  path-dependent  and  highly  interdependent.  In  both  cases,  tradi¬ 
tional  financial  options  methodologies  tend  to  fail  because  the  underlying  stochastic 
differential  equations  are  not  available  or  simply  do  not  apply  [12,26] 

A  Basic  Formulation,  Regardless  of  the  specific  framework  employed  for  decision-making, 
there  are  a  couple  of  mandatory  elements  as  part  of  any  conceivable  valuation  approach.  One 
is  the  cost  of  the  investment,  and  the  other  is  the  return  on  that  investment.  With  respect  to 
decisions  related  to  design  flexibility,  this  same  basic  approach  applies,  but  must  be  tweaked 
somewhat.  To  begin  with,  an  initial  investment  is  required  to  implement  a  more  flexible  de¬ 
sign,  which  we  refer  to  as  the  investment  cost.  Complicating  our  formulation,  however,  is  the 
fact  that  the  return  on  that  investment  is  not  directly  linked  to  the  value  of  the  flexibility.  A 
flexible  design  does  not  have  intrinsic  value',  instead,  it  is  the  concomitant  capability  asso¬ 
ciated  with  that  flexibility  that  has  value  (to  digress  the  argument  further,  it  is  really  the  mili¬ 
tary  outcome  that  can  be  achieved  via  a  given  capability  that  has  value).  Therefore,  the  value 
component  of  the  decision  formula  is  the  probabilistic  benefit  that  a  particular  capability  may 
be  realized  with  fewer  resources  (e.g.,  time  or  money)  than  had  we  not  chosen  to  make  that 
initial  flexibility  investment.  In  addition,  though,  our  notion  of  flexibility  may  require  some 
additional  cost  later  to  actually  implement  the  capability,  which  is  also  dependent  on  the 
probability  that  the  capability  will  be  effected.  Finally,  the  very  act  of  investing  in  flexibility 
(e.g.,  adding  brackets  to  a  tank  chassis)  may  adversely  affect  other  performance  attributes 
(e.g.,  speed,  maintainability,  etc)  such  that  we  need  to  include  another  value  term  to  potential¬ 
ly  decrement  lost  value  associated  with  the  flexibility  investment. 

Momentarily  setting  aside  the  aforementioned  concerns  related  to  NPV,  we  could  conceptual¬ 
ly  (and  neglecting  time-value  of  money  considerations)  formulate  the  decision  to  invest  in 
flexibility  as  follows — 

(1) 


Note  that  the  term — much  like  the  term — needs  to  include  any  value 

decrements  associated  with  adverse  consequences  to  other  performance  attributes  that  are 
incurred  by  implementing  the  capability  (e.g.,  adding  armor  may  make  a  combat  vehicle 
more  survivable,  but  will  likely  reduce  its  speed  and  maintainability). 

Assuming  that  all  terms  are  commensurable  measures  (i.e.,  monetized),  this  formulation  indi- 
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cates  that  if  the  NPV  is  greater  than  zero,  the  investment  in  the  flexible  design  option  is  worth 
pursuing.  While  eonstrueting  equation  (1)  is  relatively  straightforward,  assigning  values  to 
each  of  these  terms  is  where  the  challenge  arises.  The  two  cost  terms,  while  seldom  trivial, 
are  likely  the  most  easily  obtained,  as  we  have  ample  experience  in  estimating  the  eost  of 
engineering  solutions.  Establishing  a  valid  probability  term  is  more  diffieult  as  it  is  inherently 
linked  to  uncertainty;  however,  it  at  least  ean  be  rationally  estimated.  The  real  dilemma  is 
assoeiated  with  the  value  terms,  which  are  extremely  diffieult — if  not  impossible — to  mea¬ 
ningfully  quantify  in  the  context  of  defense  aequisition.  This  is  the  erux  of  the  problem. 

Value  of  Capability.  In  order  to  make  meaningful  value  judgments,  we  must  establish  a  utili¬ 
ty  funetion  that  will  quantify  eapability  in  some  ratio-level  units  of  comparable  value.  While 
this  is  relatively  routine  for  profit-driven  commercial  systems,  it  will  neeessarily  be  more 
challenging  for  military  systems,  as  the  utility  function  will  almost  certainly  not  involve  a 
monetizable  metrie  like  earnings.  Instead,  for  example,  we  would  need  to  somehow  devise  a 
funetion  (or  more  likely,  a  series  of  functions)  for  determining  the  utility  of  an  extremely 
wide  range  of  military  capabilities,  such  as  being  able  to  resist  jamming  or  increase  an  air¬ 
plane’s  top  speed  from  Mach  2.0  to  Mach  2.5. 

In  principle,  there  is  a  solution.  Under  the  neoelassie  economic  definition  of  value,  an  item’s 
value  can  be  established  by  determining  a  customer’s  willingness  to  pay.  Thus,  we  ean  sur¬ 
mise  that  the  value  of  a  particular  military  capability  can  be  determined  by  aseertaining  the 
maximum  amount  the  government  is  willing  to  give  up  (of  some  measureable  resource)  to 
obtain  the  capability  (i.e.,  the  value  of  a  given  capability  to  the  government  =  the  maximum 
cost  the  government  is  willing  to  pay  for  the  capability).  The  devil  is  in  the  details,  however. 
Assigning  a  numerical  value  to  the  right  side  of  this  equation  is  a  daunting  endeavor.  The 
most  obvious  approaeh  would  be  to  use  the  dollar  amount  budgeted  by  the  government.  But 
this  is  problematic  for  a  multitude  of  reasons.  Consider  that  the  actual  system  eost  may  in¬ 
clude  a  number  of  other  searce  resourees  (e.g.,  time,  eritieal  skills,  facilities)  that  are  not  cap¬ 
tured  in  the  government  budget.  Teehnieally,  economic  cost  includes  the  loss  of  opportunities 
as  well,  so  we  would  also  need  to  aceount  for  the  cost  of  losing  or  vitiating  other  eapabilities 
by  virtue  of  the  fact  that  we  are  committing  resources  to  this  capability.  Once  again,  though, 
we  would  faee  the  dilemma  of  assigning  a  value  to  a  capability,  with  only  budgets  to  guide 
us,  so  our  original  problem  is  further  complicated  beeause  it  is  now  recursive. 

In  addition,  even  if  we  were  to  aecept  that  budgeted  costs  will  be  adequate,  there  is  no  reason 
to  believe  this  represents  the  maximum  cost  the  government  is  willing  to  pay.  Firstly,  the 
government  may,  in  prineiple,  be  willing  to  budget  more  for  a  partieular  capability,  but  has 
reason  to  believe  that  a  lower  amount  will  suffiee.  The  problem  is  that  the  government  gener¬ 
ally  establishes  its  program  budgets  based  on  expected  actual  costs,  not  the  pereeived  value 
of  the  program  or  resulting  eapability  set.  Seeondly,  budget  alloeation  proeesses  are  noto¬ 
riously  volatile,  subject  to  any  number  of  political  and  bureaucratic  vagaries  that  have  noth¬ 
ing  to  do  with  the  merits  of  a  particular  program  or  capability.  Thus,  one  year’s  total  budget 
alloeation  for  a  given  program  may  be  substantially  different  from  the  next  year’s  alloeation 
for  the  same  program,  though  there  was  no  ehange  in  the  pereeption  of  its  value. 

For  the  sake  of  argument,  let’s  assume  that  we  can  tolerate  a  lower  fidelity  estimate,  and  we 
can  convince  ourselves  that  the  budgeted  costs  represent  all  costs  with  sufficient  accuracy, 
and  that  these  eosts  also  represent  the  maximum  cost  the  government  is  willing  to  pay.  Unfor¬ 
tunately,  there  is  still  another  practical  obstacle  to  establishing  a  speeific  dollar  amount  cor¬ 
responding  to  the  value  of  a  eapability.  The  fact  is  that  defense  budgets  ean  rarely  be  traced 
so  cleanly  to  desired  system  capabilities,  and  eertainly  not  at  the  levels  of  precision  that 
would  be  required  to  make  this  a  viable  approaeh  for  detailed  design  decisions.  Imagine  a 
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$10.0  billion  program  to  develop  an  aireraft  with  various  eapabilities  related  to  range,  relia¬ 
bility,  speed,  maneuverability,  lethality,  etc.  The  notion  that  we  could  indicate  exactly  how 
much  of  that  $10.0  billion  investment  the  government  is  willing  to  spend  to  achieve  a  speed 
of  Mach  2.5  may  not  be  feasible,  and  is  certainly  not  the  basis  on  which  government  program 
budgets  are  allocated  or  managed  today. 

Clearly,  using  budgeting  information  to  infer  the  value  of  capabilities  is  full  of  pitfalls.  An 
alternative  approach  would  be  to  query  system  end  users  directly.  The  most  obvious  draw¬ 
back  to  this  approach  is  the  inherent  subjectivity;  even  within  a  single  user  community,  dif¬ 
ferent  users  will  perceive  the  value  of  a  given  capability  differently.  This  would  drive  a 
comprehensive  solicitation  of  all  potential  users,  in  combination  with  some  (to  be  specified) 
means  of  aggregating  and  reconciling  those  inputs.  In  addition,  each  user’s  value  input  would 
need  to  be  provided  within  the  context  of  a  resource-constrained  environment;  else  value  as¬ 
signments  would  lose  relative  meaning.  Another  potential  problem  stems  from  the  fact  that 
the  end-user  of  the  capability  who  is  most  able  to  appreciate  its  value  is,  ironically,  the  least 
likely  to  have  any  experience  with  budgeting  and  finance,  and  thus  may  not  even  be  able  to 
translate  the  mission  value  into  monetary  terms.  Similarly,  the  user  group  may  simply  have 
no  direct  insight  into  the  costs  associated  with  the  capabilities  it  has  access  to  due  to  the  na¬ 
ture  of  service/capability  relationships  among  defense  organizations. 

Even  more  fundamentally,  flexible  design  options  may  have  no  practical  meaning  to  the  user. 
Since  we  are  inherently  interested  in  the  value  of  potential  capabilities — vice  validated  capa¬ 
bilities — the  user  may  be  unwilling  and/or  unable  to  assign  any  value  to  the  capability  at  all. 
For  if  the  potential  capability  were  valued  to  any  level  of  significance,  then  it  likely  would 
have  already  been  translated  into  a  valid  system  requirement!  Finally,  many  potentially  flexi¬ 
ble  design  decisions  over  the  course  of  a  program’s  life  (particularly  those  that  pertain  to 
process  flexibility)  have  little  to  no  impact  on  end  capabilities. 

It  can  be  argued  that  by  attempting  to  employ  both  the  willingness  to  pay  and  the  user  query 
methods,  it  may  be  possible  to  obtain  a  dollar  range  that  could  serve  to  at  least  bracket  the 
value  terms.  However,  it’s  not  clear  we  could  assign  valid  confidence  values  to  this  range  or 
that  the  calculated  range  would  be  narrow  enough  to  have  practical  utility.  Therefore,  given 
the  difficulty  of  establishing  the  value  of  military  capabilities,  we  need  a  more  flexible  ap¬ 
proach  to  determine  the  value  of  flexibility. 

A  More  Flexible  Approach 

What  if  we  could  establish  the  merits  of  a  capability  without  having  to  explicitly  determine 
its  value?  This  may  be  feasible  through  a  modification  to  the  familiar  life  cycle  cost  (FCC) 
model.  The  idea  is  to  refine  current  life  cycle  cost  calculations  to  better  account  for  the  value 
of  capability  opportunities  that  are  likely  to  arise  throughout  the  life  of  a  program.  Further¬ 
more,  the  methodology  we  propose  would  be  capable  of  inherently  evaluating  design  options 
in  aggregate,  thereby  rendering  distinctions  in  capability  like  flexibility,  versatility,  and  over¬ 
capacity  as  entirely  arbitrary.  Before  proceeding  to  a  more  comprehensive  explanation,  how¬ 
ever,  it  may  be  beneficial  to  review  the  salient  aspects  of  DOD’s  current  FCC  methodology. 

Life  Cycle  Cost.  FCC  is  a  systematic  accounting  approach  for  aggregating  all  direct  and 
many  indirect  costs  for  a  given  system.  It  includes  not  just  total  acquisition  costs,  but  also 
costs  related  to  operations,  maintenance,  and  disposal.  Importantly,  FCC  also  accounts  for 
risks,  generally  either  through  sensitivity  analyses  or  through  formal  quantitative  risk  analysis 
[27].  For  large  programs,  calculating  the  FCC  is  generally  a  tedious  undertaking  involving 
substantial  time  and  effort.  But  the  outcome  is  nevertheless  generally  deemed  to  be  worth¬ 
while.  As  a  formal  measure,  life  cycle  cost  is  entirely  straightforward,  and  easily  understood 
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by  the  typical  spate  of  stakeholders,  to  include  systems  engineers,  users,  and  contractor  and 
government  managers.  Moreover,  by  providing  senior  decision-makers  with  their  single  best 
source  of  estimated  cost  to  achieve  a  given  capability,  the  LCC  is  often  instrumental  in  de¬ 
termining  the  ultimate  fate  of  a  program. 

Formal  DOD  guidance  calls  for  the  LCC  to  be  first  accomplished  as  part  of  the  initial  Analy¬ 
sis  of  Alternatives  (AoA)  and  is  only  updated  as  part  of  major  milestone  decision  reviews. 
Aside  from  these  updates,  however,  the  system  LCC  is  generally  a  static  measurement.  When 
calculated,  it  provides  a  “snapshot”  estimate  of  total  life  cycle  cost  on  the  assumption  that 
there  will  be  no  deviations  from  key  cost,  schedule,  and  performance  parameters,  which  are 
collectively  referred  to  as  the  acquisition  program  baseline  (APB)  [27].  Of  course,  one  thing 
we  know  with  near  certainty  is  that  there  will  almost  always  be  deviations  from  the  APB. 

While  the  assumption  of  a  static  APB  may  be  unwarranted,  programs  proceed  with  it  anyway, 
presumably  because  the  alternative  of  trying  to  account  for  the  non-deterministic  uncertainty 
in  precisely  how  the  program  will  deviate  from  the  APB  is  simply  not  possible,  or  at  least  just 
too  daunting.  It  can  be  argued,  however,  that  even  though  uncertainty  is — ^by  definition — not 
deterministic,  it  may  be  possible  to  employ  stochastic  probability  methods  that  can  yield  cost 
estimates  that  are  likely  to  be  more  accurate  in  the  long  run.  Although  establishing  the  initial 
models  to  accomplish  this  would  require  significant  resource  investment,  the  possibility  of 
more  accurate  LCC  estimates — and  the  improvement  in  decision-making  that  would  accom¬ 
pany  that — promises  an  enormous  return  on  such  an  investment. 

Life  Cycle  Cost  Under  Uncertainty.  Thus,  there  is  substantial  motivation  to  provide  im¬ 
proved  LCC  estimates,  at  least  to  the  level  required  to  support  decisions  considering  alterna¬ 
tive  flexible  design  options.  The  notion  that  this  can  be  done  by  accounting  for  random  events 
that  affect  the  system  forms  the  basis  of  life  cycle  cost  under  uncertainty  (also  referred  to  as 
stochastic  life  cycle  cost),  which  was  mentioned  earlier  as  part  of  the  discussion  on  value- 
driven  design.  The  idea  of  applying  this  strategy  to  acquiring  military  systems  appears  to 
have  been  first  introduced  by  Brown  in  two  papers  related  to  the  F6  satellite  program  [14,28]. 
As  described  by  Brown,  stochastic  life  cycle  cost  is  premised  on  three  assertions. 

•  The  cost  to  develop,  procure,  and  operate  a  system  with  some  assured  minimum  ca¬ 
pability  over  its  lifecycle  is  not  a  deterministic  value. 

•  Instead,  this  cost  can  be  modeled  as  a  random  variable  with  a  probability  distribution 
resulting  from  a  set  of  uncertainties  introduced  throughout  the  system's  life. 

•  This  random  variable  metric  is  a  relevant  basis  for  comparison  between  alternative 
system  architectures  and  design  choices. 

Brown  is  to  be  commended  for  introducing  this  simple  but  deceptively  powerful  notion  of 
stochastic  life  cycle  cost.  However,  the  initial  treatment  does  not  develop  the  principle  fully, 
nor  explore  its  broader  applicability.  The  type  of  stochastic  events  he  considers  are  only  those 
specific  events  that  critically  influence  the  success  of  a  satellite  system,  i.e.,  launch  failure 
and  on-orbit  component  failure.  Brown  explicitly  does  not  consider  other  aspects  of  life  cycle 
uncertainty  that  affect  virtually  all  programs,  such  as  “requirements  creep,  funding  stream 
volatility,  technology  development  risk,  and  volatility  of  demand”  [14].  Yet  he  clearly  does 
recognize  that  the  model  could  be  applied  to  these  other  sources  of  uncertainty,  noting  that 
these  variables  are  “left  for  future  analysis.”  To  date,  it  does  not  appear  that  such  an  analysis 
has  been  accomplished  by  him  or  others. 

Consequently,  we  propose  a  research  strategy  to  logically  extend  this  promising  technique  in 
a  manner  that  may  provide  a  number  of  potential  benefits  over  current  practices.  Specifically, 
we  intend  to  expand  the  life  cycle  cost  under  uncertainty  idea  to  a  robust  and  comprehensive 
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methodology  for  effectively  valuing  system  design  alternatives.  For  the  remainder  of  this 
paper,  we  explore  how  such  an  approach  could  be  applied  to  uncertainty  as  related  to  system 
performance.  We  expect  to  address  its  applicability  to  other  sources  of  uncertainty  (i.e.,  cost 
and  schedule)  in  subsequent  efforts. 

Another  modification  to  enhance  the  utility  of  the  LCC  concept  is  that  it  should  not  be 
viewed  as  simply  a  static  measure  only  to  be  crafted  in  support  of  key  milestones.  Just  as 
LCC  is  an  essential  decision  tool  for  those  in  the  role  of  Milestone  Decision  Authority 
(MDA)  and  above  to  gauge  the  value  of  a  program,  it  can  fulfill  the  same  principal  function 
to  those  who  serve  at  the  program  manager  level  and  below.  Moreover,  estimates  of  life  cycle 
cost  are  not  useful  just  periodically,  but  have  ongoing  utility  at  all  stages  of  the  program,  as 
design  decisions  are  continually  required  at  various  levels  of  the  program  which  (to  varying 
degrees)  are  likely  to  impact  the  overall  system  cost.  And  whereas  early  LCC  values  would 
naturally  be  focused  on  high-level  architectural  decisions,  as  the  program  matures,  and  the 
requirements  baseline  migrates  from  functional  to  allocated  to  product,  the  decision  trade 
space  will  concomitantly  shift  to  the  more  detailed  design  implementations.  Thus,  this  dy¬ 
namic  and  (probabilistically)  more  accurate  LCC  should  arguably  be  managed,  updated,  and 
referenced  as  often  as  the  program  schedule. 

Current  Expected  Value  Life  Cycle  Cost,  To  capture  the  utility  of  this  improved  LCC  con¬ 
cept,  we  offer  the  appellation,  CEVLCC,  which  stands  for  Current  Expected  Value  Life  Cycle 
Cost.  The  name  is  intended  to  convey  a  couple  of  key  distinctions  from  the  standard  LCC  and 
Brown’s  stochastic  LCC.  The  “Expected  Value”  phrase  discriminates  CEVECC  from  the 
standard  ECC  as  a  more  probabilistically  accurate  measurement  of  system  cost;  whereas  the 
word  “current”  is  intended  to  connote  the  fact  that  the  CEVECC  would  be  employed  as  a 
living,  continually  updated  decision  analysis  tool.  The  notion  that  an  ECC  estimate  might  be 
applied  dynamically,  and  at  lower  levels  of  system  design,  is  distinct  from  Brown’s  view  that 
the  stochastic  ECC  could  only  be  useful  for  “preliminary  trade  space  exploration”  and  not  for 
value  determinations  “below  the  architectural  level”  [28]. 

Eor  clarity,  here  are  the  specific  assumptions  that  must  hold  for  this  approach  to  be  valid — 

1 .  As  programs  mature,  there  will  be  unpredictable  deviations  from  the  APB  that  affect 
the  system’s  ECC 

2.  It  is  possible,  on  average,  to  provide  a  more  accurate  ECC  estimate  through  probabil¬ 
istic  modeling  of  the  stochastic  processes  that  cause  deviations  in  the  APB 

3.  The  cost  of  the  effort  required  to  calculate  a  more  accurate  ECC  is  more  than  offset  by 
the  value  obtained  by  the  more  accurate  ECC 

4.  Given  the  CEVECC  cost  accounting  methodology,  as  long  as  each  design  meets  all  of 
its  threshold  requirements,  then  its  relative  value  can  be  inferred  from  its  cost 

In  addition,  the  proposed  methodology  is  straightforward,  consisting  of  the  following  steps: 

•  Establish  system  design  options 

•  Construct  time-phased  probability  distribution  functions  (PDEs)  associated  with  all 
existing  key  cost,  schedule,  and  technical  performance  parameters  of  the  program 

•  Assign  time-phased  probabilities  for  potential  new  capabilities  of  the  system 

•  Estimate  standard  (i.e.,  traditional)  ECC  cost 

•  Estimate  costs  associated  with  modifications  (consistent  with  PDEs)  to  baseline  cost, 
schedule,  and  technical  performance  parameters 

•  Estimate  costs  associated  with  the  addition  of  new  capabilities 

•  Calculate  CEVECC  for  each  system  design  option  and  select  alternative  associated 
with  the  lowest  CEVLCC 
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Hypothetical  Use  Case,  To  appreciate  the  process  and  potential  utility  of  CEVLCC,  we  illu¬ 
strate  its  application  using  a  hypothetical  missile  defense  scenario.  For  simplicity,  we  will 
only  consider  technical  performance  as  part  of  this  analysis. 

Assume  we  have  a  requirement  to  protect  a  high-value  facility  in  a  sensitive  overseas  loca¬ 
tion,  which  must  conform  to  the  following  four  Key  Performance  Parameters  (KPPs). 


Table  1:  KPPs  for  Missile  Defense  Scenario 


# 

Key  Performance  Parameter 

Threshold 

Objective 

1 

Protect  facility  from  ballistic  missile  attack  with  X%  assurance 

X=95 

X=99 

2 

Engage  only  missiles  if  >  X%  confident  they  represent  an  immi¬ 
nent  threat  to  the  facility 

X=90 

X=95 

3 

No  evidence  of  military  presence  w/in  X  mi.  of  facility 

X=25 

X=40 

4 

Be  able  to  engage  X  missile(s)  simultaneously 

X=1 

X=5 

The  second  CEVLCC  assumption  states  that  it  is  possible  to  formulate  probabilistic  modeling 
of  the  stochastic  processes  that  cause  deviations  in  the  APB.  One  way  to  accomplish  this  is  to 
treat  the  value  for  each  performance  parameter — in  this  case,  each  threshold  KPP  value — as  a 
random  variable,  and  construct  the  probability  function.  To  do  this  with  any  semblance  of 
confidence  would  likely  require  extensive  empirical  data  from  a  variety  of  different  require¬ 
ment  categories,  program  types,  program  levels,  acquisition  strategies,  etc.  Furthermore,  the 
PDFs  would  be  valid  only  at  a  single  point  in  time,  so  they  would  need  to  be  revised  as  the 
program  matures  and  new  information  becomes  available.  Clearly,  construction  and  mainten¬ 
ance  of  the  CEVLCC  would  require  significant  effort;  nevertheless,  it  could  be  done,  and 
would  be  worth  doing  if  the  third  assumption  above  holds.  Figure  1  below  provides  examples 
of  what  those  PDFs  might  look  like  in  the  case  of  KPPs  #1  and  #3: 


KPP#1 


Threshold  Value 


_  KPP  #3 

0.08  -1 - - - - 


0.06 

ra  0.04 

O 

“■  0.02 


0.00  -1 - - - - - - - 

25  30  35  40  45  50  55  60 

Threshold  Value 


Figure  1 :  Notional  PDFs  of  Missile  Defense  Scenario  KPPs 


In  both  cases,  the  x-axis  is  the  random  variable  (i.e.,  the  KPP  threshold  value),  and  the  y-axis 
is  the  probability  associated  with  a  particular  value  of  the  random  variable.  For  simplicity,  we 
have  chosen  not  to  depict  the  probability  that  the  variable  will  remain  the  same  or  decrease, 
but  in  a  comprehensive  model,  these  probabilities  would  likely  need  to  be  determined  as  well. 

After  establishing  the  PDFs  for  the  parameter  values  of  existing  capabilities,  we  next  need  to 
account  for  the  probability  that  the  system  will  be  required  to  support  new  (and  obviously 
foreseeable)  capabilities.  For  instance,  we  might  conceive  of  the  following  two  potential  new 
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capabilities,  along  with  their  estimated  likelihoods: 

•  Proteet  against  eruise  missile  threats  (15%) 

•  Proteet  against  unconventional  ordnanee  attacks  (e.g.,  suicide  bomber)  (2%) 

Eaeh  of  these  probability  funetions  will  require  a  temporal  dimension  as  well.  In  other  words, 
these  estimations  of  probability  are  assoeiated  with  a  given  time  horizon,  and  will  neeessarily 
vary  depending  on  that  horizon.  For  this  seenario,  we  might  estimate  that  if  a  requirement 
related  to  the  first  new  eapability  (i.e.,  proteet  against  eruise  missiles)  has  not  been  introdueed 
by  the  Preliminary  Design  Review  (PDR),  then  its  likelihood  of  being  imposed  between  the 
PDR  and  the  Critieal  Design  Review  (CDR)  is  three  pereent,  and  its  likelihood  of  being  im¬ 
posed  between  the  CDR  and  the  Test  Readiness  Review  (TRR)  is  one  pereent,  and  so  on. 
Viewed  in  this  way,  we  reeognize  a  certain  similarity  between  these  various  PDFs  and  tradi¬ 
tional  risk  burn-down  plans.  This  is  an  important  point,  as  the  PDFs  would  need  to  be  ma¬ 
naged  in  a  similar  manner,  and  eould  reasonably  be  integrated  with  traditional  risk  analyses. 

In  both  cases  (i.e.,  the  modification  of  existing  capabilities  and  the  addition  of  new  eapabili- 
ties),  the  assigned  probabilities  will  admittedly  be  estimates,  perhaps  quite  rough  ones.  Will 
they  be  exaetly  right?  Absolutely  not.  If  our  stoehastie  models  are  at  all  valid,  are  they  likely 
to  be  closer  to  reality  than  the  assumption  that  nothing  will  ehange  over  the  remaining  life  of 
the  program?  Almost  eertainly. 

Next  is  the  cost  assessment  step.  This  is  exeeuted  in  the  eontext  of  whatever  design  options 
we  have  available  to  us  at  any  given  time.  Fet’s  assume,  based  on  earlier  assessments,  that 
the  program  has  ehosen  a  defensively-oriented  arehiteeture  that  engages  ballistie  missile 
threats  during  the  terminal  phase.  Fike  all  true  deeisions,  the  program  has  made  an  irrevoea- 
ble  alloeation  of  resourees  as  a  result,  and  has,  to  some  extent,  neeessarily  constrained  their 
design  spaee  going  forward.  Nevertheless,  the  eommitment  to  the  terminal  phase  option  still 
leaves  a  number  of  fundamental  design  deeisions  open  to  them.  We  then  postulate  the  follow¬ 
ing  list  of  arehiteetural  possibilities  being  eonsidered  by  the  program: 

1 .  Terrestrial  intereeptor  system  stationed  at  least  25  miles  from  faeility 

2.  Concealed  (e.g.,  underground)  terrestrial  intereeptor  system 

3.  Airborne  interceptor  system 

4.  Terrestrial  direeted-energy  system  stationed  at  least  25  miles  from  faeility 

5.  Coneealed  (e.g.,  underground)  terrestrial  direeted-energy  system 

6.  Airborne  direeted-energy  system 

7.  Hardened  strueture  that  ensures  survivability  of  facility 

8.  Force  field 

Each  of  these  architeetures  has  relative  strengths  and  weaknesses  based  on  the  KPPs  as  writ¬ 
ten.  And  each  of  these  designs  has  its  own  inherent  costs  to  implement.  All  else  being  equal, 
under  the  traditional  eoneeption  of  FCC,  the  option  above  with  the  lowest  FCC  that  also 
meets  all  threshold  requirements  is  typieally  the  one  that  will  be  seleeted.  This  is  the  erux  of 
the  problem,  as  this  traditional  approaeh  does  not  aeeount  for  the  value  of  the  flexibility  em¬ 
bedded  within  certain  arehiteetural  options. 

The  CEVFCC,  however,  requires  that  additional  cost  estimating  be  performed  against  the 
range  of  potential  new  KPP  threshold  values  as  well  as  the  potential  new  eapabilities.  Clearly, 
some  of  these  options  are  better  poised  to  aeeommodate  ehanges  in  the  KPP  thresholds.  For 
instanee,  the  eoneealed  terrestrial  arehiteetures  (i.e.,  options  #2  and  #5)  will  have  no  addi¬ 
tional  eost  should  there  be  an  increase  associated  with  the  threshold  value  of  KPP  #3,  whe¬ 
reas  the  non-eoneealed  versions  (i.e.,  options  #1  and  #4)  would  likely  have  an  enormous  eost 
impaet.  Similarly,  some  arehiteetures  ean  more  easily  aeeommodate  new  eapabilities.  If  the 
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program  is  directed  to  incorporate  the  capability  to  protect  the  facility  against  cruise  missiles, 
then  the  airborne  interceptor  system  can  be  modified  much  more  easily  than  the  underground 
interceptor  system  (i.e.,  the  airborne  system  is  more  flexible).  And  the  hardened  structure 
option  will  not  have  to  be  modified  at  all,  as  the  capability  to  withstand  the  cruise  missile 
strike  was  already  embedded  in  its  design  (i.e.,  it’s  overcapacitized). 

Once  we’ve  determined  the  estimated  costs  for  the  potential  changes  to  the  system,  we  calcu¬ 
late  all  of  the  expected  values  for  each  design  with  respect  to  each  change.  So  suppose  that 
for  all  three  directed  energy  architectures,  we  estimated  the  following  additional  costs  for  the 
potential  range  of  changes  in  the  value  of  the  KPP#1  threshold. 


Table  2:  Marginal  Probability  Costs  for  Directed  Energy  Architectures 


Index (  ) 

KPP#1  Threshold 
(X) 

Additional  Cost  to 
Implement  (  ) 

Probability  (from 
Figure  1) 

1 

96.0% 

$0.0M 

10.0% 

2 

96.5% 

$0.0M 

7.0% 

3 

97.0% 

$0.0M 

5.0% 

4 

97.5% 

$1.0M 

3.5% 

5 

98.0% 

$1.0M 

2.5% 

6 

98.5% 

$1.0M 

1.7% 

7 

99.0% 

$3.0M 

1.1% 

8 

99.6% 

$6.0M 

0.5% 

9 

99.9% 

$20.0M 

0.1% 

Using  the  standard  formula  for  expected  value,  then  architectures  #4,  #5,  and  #6  (i.e.,  the 
directed  energy  architectures)  have  an  expected  value  of  $160k  with  respect  to  KPP  #1.  We 
repeat  this  process  for  each  architecture  for  the  remaining  KPPs.  We  then  account  for  the 
potentially  new  capabilities  in  the  same  manner,  although  the  expected  value  calculation  is  a 
trivial  weighted  probability  with  a  single  term.  So  for  each  architecture,  a  separate  CEVLCC 
is  calculated  by  summing  its  baseline  LCC,  its  summation  of  expected  values  for  the  modifi¬ 
able  capabilities,  and  its  summation  of  expected  values  for  the  new  capabilities,  i.e. — 


(2) 


This  leads  to  the  fourth  assumption.  As  long  as  each  design  meets  all  threshold  requirements, 
then  its  relative  value  can  be  inferred  from  its  cost.  Ordinarily,  this  would  not  be  valid,  but 
given  our  cost  formulation  methodology,  we  implicitly  accounted  for  those  discriminators 
that  would  otherwise  have  contributed  to  the  value  side  of  the  equation.  Specifically,  there  is 
no  need  to  assess  how  much  to  “credit”  a  particular  design  for  its  ability  to  exceed  a  KPP 
threshold  or  its  capacity  to  accommodate  future  changes.  Both  of  these  inherent  design  values 
are  captured  (albeit  in  complementary  fashion)  in  our  marginal  probability  cost  estimates 
within  CEVLCC.  In  other  words,  if  a  particular  design  option  were  able  to  more  inexpensive¬ 
ly  accommodate  a  capability  change — whether  via  flexibility  or  via  overcapacitization — its 
weighted  cost  would  be  less  than  the  competing  designs,  and  it  value  would  be  greater.  Note 
that  this  is  why  it  is  only  valid  to  compare  systems  that  meet  all  threshold  requirements,  as 
there  needs  to  be  a  value  baseline  to  reference.  Based  on  these  assumptions,  then,  we  can  now 
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assert  that  the  system  that  is  the  best  value  is  simply  the  one  with  the  lowest  CEVLCC. 

As  promising  as  CEVLCC  might  be,  we  reeognize  there  are  also  a  number  of  potential  draw- 
baeks  to  this  technique,  most  of  which  are  tied  to  the  model  assumptions.  For  instance,  the 
fundamental  nature  of  defense  acquisition  may  be  more  chaotic  than  stochastic,  thus  prevent¬ 
ing  accurate  predictive  modeling  over  a  reasonable  time  horizon,  and  fully  precluding  analy¬ 
sis  of  unforeseeable  changes  (violation  of  assumption  2).  Also,  to  be  most  effective, 
CEVLCC  would  need  to  be  comprehensive  and  current,  which  results  in  a  large  number  of 
permutations  to  account  for,  thus  potentially  making  its  implementation  cumbersome.  Even  if 
the  resource  investment  is  deemed  to  be  worthwhile  very  early  in  the  program  (i.e.,  when 
design  decisions  are  most  impacting),  it  is  possible  that  the  return  on  investment  will  not  be 
sufficient  to  justify  its  use  much  further  into  the  program  (violation  of  assumption  3).  Impor¬ 
tantly,  CEVLCC,  as  currently  conceived,  also  cannot  effectively  provide  a  relative  evaluation 
of  design  options  that  do  not  meet  threshold  requirement  levels  (violates  assumption  4).  Fi¬ 
nally,  the  CEVLCC  does  not  entirely  sidestep  the  problem  of  valuing  capability,  as  excess 
capability  above  the  threshold  often  does  have  value  that  must  be  accounted  for.  This  tech¬ 
nique  does  not  properly  account  for  the  temporal  benefit  that  an  overcapacitized  solution  pro¬ 
vides,  i.e.,  having  a  newly  desired  capability  immediately  (or  more  quickly)  available  vice 
waiting  for  development  and  implementation. 

Conclusion 

There  is  consensus  that  uncertainty  is  a  principal  reason  that  DOD  programs  continue  to 
struggle  mightily  with  respect  to  their  ability  to  adhere  to  cost  and  schedule  projections. 
While  acquisition  policies  and  strategies  that  aim  to  abate  uncertainty  are  admirable  and  often 
useful,  ultimately  they  can  only  help  so  much.  Since  uncertainty  is  a  certainty,  programs  may 
be  better  served  by  infusing  their  systems  with  an  inherent  ability  to  effectively  respond  to 
uncertainty.  The  singular  term  most  commonly  associated  with  such  an  ability  is  flexibility. 
While  flexibility  is  arguably  the  single  best  term  for  this  concept,  even  it  is  insufficient  to 
capture  the  full  range  of  capability  responsiveness  we  would  like  our  systems  to  have.  We 
may  also  need  them  to  be  versatile  and/or  overcapacitized.  However,  making  a  system  flexi¬ 
ble,  versatile,  or  overcapacitized  inevitably  requires  additional  investment  that  must  be  justi¬ 
fied.  The  only  viable  way  to  provide  that  justification  is  to  quantify  the  value  of  the 
capabilities  that  can  be  more  easily  achieved  because  of  the  investment.  For  military  weapons 
systems,  this  task  is,  at  best,  extremely  challenging,  and,  at  worst,  simply  not  feasible. 

Consequently,  a  fundamentally  different  approach  is  needed — one  that  does  not  rely  on  an 
explicit  valuation  of  potential  capabilities,  and  is  capable  of  evaluating  design  options  more 
strategically,  thus  shifting  the  focus  from  the  somewhat  narrow  view  of  just  flexibility,  per  se, 
to  the  broader  view  of  capabilities,  regardless  of  how  they  are  achieved.  Thus,  we  propose  the 
CEVLCC,  a  top-down,  intrinsic  value  model  based  on  the  familiar  notion  of  life  cycle  cost. 
The  idea  is  premised  on  the  notion  that  the  need  for  capability  changes  in  a  program  arises  in 
a  stochastic  manner  that  can  be  modeled  and  incorporated  into  a  continually  updated,  ex¬ 
pected  value  model  of  total  program  cost.  We  believe  CEVLCC  potentially  offers  a  number 
of  advantages  over  current  practices — 

•  An  inherent  focus  on  capability  in  toto  that  serves  to  automatically  assimilate  relevant 
capability  concepts,  such  that  discriminatory  design  considerations  like  overcapacity 
and  versatility  become  irrelevant 

•  An  inherent  ability  to  incorporate  cost  and  schedule  components  of  a  program,  there¬ 
by  obviating  the  distinction  between  design  flexibility  and  process  flexibility 

•  Being  comprised  of  concepts  already  familiar  to  the  acquisition  community  (i.e.,  life 
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cycle  cost  and  risk  analysis),  thereby  greatly  redueing  cultural  entry  barriers 

•  Having  a  simple  premise  and  an  intuitive  output  (i.e.,  cost),  both  of  which  encourage 
adoption  among  stakeholders  aeross  the  acquisition  community 

•  Not  being  subjeet  to  critieisms  specific  to  real  options  analysis 

•  Being  able  to  mostly  sidestep  theoretieal  and  practical  challenges  assoeiated  with  va¬ 
luing  military  capabilities 

Currently,  the  CEVLCC  concept  is  largely  notional,  and  significant  research  effort  remains  to 
determine  its  validity  and/or  utility.  Most  of  the  near-term  work  is  intended  to  validate,  or  at 
least  characterize  the  limitations  of,  the  CEVLCC  assumptions.  Specifieally,  we  plan  to — 

•  Analyze/characterize  APB  behavior  for  historieal  programs  and  examine  concomitant 
LCC  behavior  with  the  intent  of  identifying  salient  factors  that  drive  perturbations. 

•  Construct  a  basic  CEVLCC  model  based  on  these  salient  faetors. 

•  Compare  the  LCC  aecuraey  for  historical  programs  over  time  to  the  corresponding 
CEVLCC,  and  conduct  tradeoff  analyses  to  determine  when  the  return  on  investment 
in  the  CEVLCC  model  is  no  longer  worthwhile. 

•  Identify/develop  alternate  methodology  to  address  CEVLCC  weakness  with  respect 
valuing  existing  excess  capability. 

•  Refine  CEVLCC  model  and  validate  via  historieal  case-studies 
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